A reciprocal peer review system to support college students' writing

As students’ problem-solving processes in writing are rarely observed in face-to-face instruction, they have few opportunities to participate collaboratively in peer review to improve their texts. This study reports the design of a reciprocal peer review system for students to observe and learn from each other when writing. A sample of 95 undergraduate students was recruited to construct texts with the support of web-based reciprocal peer review in the processes of modelling, coaching, scaffolding, articulation, reflection and exploration. The results of the study revealed that these six processes helped students externalise and visualise their internal writing processes so that they could observe and learn from peers in writing as well as support peers in making text revisions. During their extensive and reciprocal interactions with various peers, students addressed mutual concerns in each other’s text revisions. They constructed collaborative language knowledge for text improvement as local revisions (grammatical corrections) and global revisions (corrections on the development, organization or style of a text) were made in their final texts. The students’ perceptions towards text improvement in this web-based peer review of modelling, coaching, scaffolding, articulation, reflection and exploration are also discussed in this study. Introduction Improving writing skills is important for college students since ‘professional and academic success in all disciplines depends, at least in part, upon writing skills’ (Cho & Schunn, 2007, p. 409). Many studies have emphasised that writing is no longer regarded as an individual task but rather may be supported by feedback from peers to improve the text (eg, Liou & Peng, 2009; Lundstrom & Baker, 2009; Min, 2006; Vass, Littleton, Miell & Jones, 2008). Students can benefit more from different peers if they have gone through similar writing situations and difficulties in providing helpful suggestions and revisions (van Leeuwen, Tiesinga, Jochemsen & Post, 2009). Through the reciprocal process of peer review in writing, students are able to develop new ideas and perspectives as well as improve their writing skills (Lundstrom & Baker, 2009). Different from face-to-face writing instruction, web-based peer review allows students to visualise their thinking and behaviors in solving problems of writing, where the thinking process is complicated and not easily observed (Liu, 2005; Patel, Kinshuk & Russel, 2003). Through web-based peer review, students are able to observe each peer’s writing processes recorded online and engage in interaction to receive peers’ assistance or to provide scaffolding to other peers for text improvement. In the pursuit of text improvement, they not only benefit from receiving feedback of peers but also from providing assistance to them (Cheng & Ku, 2009; Duran & Monereo, 2005). British Journal of Educational Technology Vol 42 No 4 2011 687–700 doi:10.1111/j.1467-8535.2010.01059.x © 2010 The Author. British Journal of Educational Technology © 2010 Becta. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. However, previous studies have shown several drawbacks to peer review in writing. First, the rate of adopting peers’ revision is low when peers engage in non-revision-oriented feedback such as chatting and making complimentary remarks, rather than revision-oriented feedback such as suggestions, clarifications and critical evaluations (Liou & Peng, 2009). Second, peer review in writing is restricted to the lexical level for students often fail to participate in the discussion of text content or organisation (Cho & Schunn, 2007; Lundstrom & Baker, 2009; Storch, 2005). Third, students have few opportunities to reflect upon peer review as they cannot compare the differences between the different feedback from various peers (van Leeuwen et al, 2009; Vass et al, 2008; Xiao & Lucking, 2008). Fourth, students will not benefit from peer review until they are asked to rewrite their texts based on peers’ revision (Braine, 1997; Paulus, 1999). Thus, some important issues need to be addressed to ensure the quality and effectiveness of peer review in writing. Web-based peer review processes The development of the web-based peer review processes described in this study is based on the theory of cognitive apprenticeship where six processes are included for externalising knowledge— modelling, coaching, scaffolding, articulation, reflection and exploration (Collins, Brown & Newman, 1989). As writing is usually combined with complex internal processes, how to transform one’s thinking into a visible form is important in web-based peer review. Through the web-based peer review processes conducted in this study, students were able to observe peers’ writing process, provide revisions both at the lexical and discourse level, communicate with peers, compare one’s own texts with those revised by peers and decide whether to accept or reject peers’ suggestions. In the process of modelling, all of students’ texts are presented in the online system for students to observe how their peers read and revise others’ texts. With modelling, students can learn reviewing skills more effectively without repeated struggles of trial and errors (Wang & Bonk, 2001; Williams, 1992). In addition, students have to be more responsible in revising others’ texts because their reviews could also be models for others (Yoshimura, 2009). In the process of coaching, students read and revise peers’ texts and provide local revisions (corrections of grammatical errors). ‘Coaching may serve to direct students’ attention to a previously unnoticed aspect of the task or simply to remind the student of some aspect of the task that is known but has been temporarily overlooked’ (Collins et al, 1989, p. 481). It is reported that students do not detect their own grammatical errors in writing (Cho & Schunn, 2007; Li, 2006). In this study, peers’ local revisions are considered as coaching as these errors are generally unnoticed by students and regarded as minor mistakes in writing (Cho & Schunn, 2007; Li, 2006). Different from the process of coaching which refers to peers’ direct corrections on the texts, scaffolding refers to peers’ suggestions for student writers to reorganise their texts by themselves. This support helps students manage complex tasks (Chee, 1995). In this study, global revisions (corrections to the development, organisation or style of a text) are considered as scaffolding as they enable students to reexamine and reorganise their texts in terms of development, organisation or style. This may result in a student writer’s deletion or addition of a new paragraph or the reorganisation of an entire text. Thus, global revisions are not limited to the lexical level but rather expand upon discourse level. Both local and global revisions are important to students’ text improvement (Cho & Schunn, 2007; Li, 2006). Articulation involves any method of encouraging students to express their knowledge, reasoning or problem-solving processes in peer review. For example, students are encouraged not only to make revisions but also to provide reasons for revisions. Such tasks require students to participate in generating language knowledge and evaluating the writing outcomes. In reflection, students compare their own problem-solving processes in writing with those of peers (Liu, 2005). Such 688 British Journal of Educational Technology Vol 42 No 4 2011 © 2010 The Author. British Journal of Educational Technology © 2010 Becta. comparisons aid students in diagnosing their difficulties and adjusting their revision strategies until they achieve the goal of text improvement. Exploration enables students to become independent learners as they are encouraged to select appropriate and effective peer review for improving their own texts. For example, students are encouraged to revise their texts into final drafts based on their evaluation of the feedback from peers. As a result, students are expected to become more active learners. Purpose of this study The purpose of this study is to engage college students in reciprocal peer review for text improvement using an online system. Participants in the system are encouraged to externalise their thinking throughout the entire process of peer review in writing. Among the reciprocal interactions with peers, the focus is placed on the processes of peer review, because the students’ decisions for making text revisions can be observed (Hansen & Liu, 2005; Storch, 2005). This focus differs from many studies whose emphasis is on the students’ written products. In this paper, three research questions are addressed: (1) how do the processes of peer review improve students’ texts in the online system?, (2) how do students receive support from and provide support to peers in web-based peer review?, (3) how do students perceive their text improvements in web-based peer review? Method To support college students in reciprocal peer review, an online system was designed for them to experience the processes of modelling, coaching, scaffolding, articulation, reflection and exploration. In the system, students followed each process in peer review with a common goal of text improvement. Participants Three learning English as a Foreign Language (EFL) writing classes were randomly selected from a university of science and technology in central Taiwan. In these three writing classes, there were 95 students with a common background in two respects: (1) all of them had passed the intermediate level of General English Proficiency Test, a nationwide screening test, administered by the university in the selection of students who wish to major in English; and (2) they had taken the same writing class for 2 years in this university and were in the third year of their studies. The objective of these three writing classes was to develop students’ writing skills through the support of peer review. Students were randomly assigned a user identifier (ie, P1–P95) in order to be anonymous in the online system. They were encouraged to interact with their peers by revising peers’ texts and evaluating peers’ suggestions before rewriting their first drafts. System development Designing an extensive and reciprocal learning environment is important to promote peer review in writing (Patel, Kinshuk & Russel, 2003; Woolley & Jarvis, 2007). Six processes—modelling, coaching, scaffolding, articulation, reflection and exploration—were implemented in the system to encourage students’ thinking and problem-solving processes in writing. Each process is described as follows. 1. Modelling: Some students might have difficulties in writing a text that their teacher has assigned. One possible way to overcome their difficulties is to read peers’ texts before posting their own texts in the system. As shown in Figure 1, the number of clicks made when reading each individual student’s text is presented in the system; students are also allowed to use the function key ‘My Collection’ to collect their favorite peers’ texts for modelling. That is, peers’ texts can serve as models for them to develop and organise their own texts on the same line. A reciprocal peer review system 689 © 2010 The Author. British Journal of Educational Technology © 2010 Becta. 2. Coaching: A list of error types in local revision is provided by the system for students to select when checking peers’ texts (Figure 2). The definitions and examples for each error type are also provided in the system for students to look up. After the students’ selection of error types, their local revisions are automatically highlighted with a number and a triangle icon (Figure 3). With peers’ coaching in local revisions, students can pay attention to the grammatical errors that they have not noticed. 3. Scaffolding: A list of error types in global revision is provided by the system for students to refer to (Figure 4). A definition and example of each error type is also presented for students to look up. 4. Articulation: In the system, students are allowed to discuss each revision in a dialogue box. In the dialogue box (Figures 5 and 6), peers’ selected error type and explanations for a local or global revision are presented. Student writers then evaluate the revision and give feedback to peers’ comments. Through articulation, students externalise their knowledge and thinking process in making text revisions. 5. Reflection: Students can use the Diff Engine embedded in the system to compare their texts with the ones revised by peers. As shown in Figure 7, the differences between the two selected texts are detected and highlighted by boldface and crossing-out. While reflecting on peers’ revision, students decide whether to accept it or not. With reflection, students can acquire multiple perspectives from peers on the same topic. 6. Exploration: In order to become independent writers, students are encouraged to revise their first drafts into their final drafts based on their evaluation of peers’ comments. Title Author Date & Time No. of Clicks in reading the text My Collection Literacy (Literacy in the United States) P16 2008-11-23 21:17:15 59 6 HW 14: The importance of literacy (Literacy in the United States) P03 2008-11-23 20:30:17 57 5 HW14: Unclear definition of literacy (Literacy in the United States) P01 2008-11-23 20:10:34 82 7 Figure 1: Peers’ texts serve as model texts in the system Singular and plural error Spelling error Erroneous part of speech ... Run-on sentence Fragment Others Run-on sentence The night was cold we forgot to bring our coats. (X) The night was cold, and we forgot to bring our coats. (O). Figure 2: Error types in local revision Everyone wants to improve [1] hishim or her knowledge. Figure 3: A local revision highlighted in the system 690 British Journal of Educational Technology Vol 42 No 4 2011 © 2010 The Author. British Journal of Educational Technology © 2010 Becta. Different versions of a revised text are shown in the system. These enable students to raise their awareness to take steps in improving their texts. After posting their final drafts, students were also asked to evaluate their revisions for the final drafts by reporting their reasons for revision. With evaluation, students are responsible for their text revision and improvement. Students’ actions in each of the six processes of peer review are recorded in the trace result of the system. This is available not only for the teacher to monitor students’ actions in the system, but also for students to visualise their writing processes. From the trace result, the process of peer review for text improvement can be examined. Perspective Article Construction Organization Clarity of Purpose Development of Purpose Inter-paragraph Transition Correctness in the Standard Conventions Others Clarity of purpose The main ideas delivered were clear and precise. Figure 4: Error types in global revision Everyone wants to improve [1] hishim or her knowledge.